Educ. Sci. 2014, 4, 193-208; doi:10.3390/educsci4020193 


OPEN ACCESS 


education 

sciences 

ISSN 2227-7102 

www.mdpi. com/j ournal/education 

Article 

The Effects of Facilitating Feedback on Online Learners’ 
Cognitive Engagement: Evidence from the Asynchronous 
Online Discussion 

Wenge Guo Ye Chen * 2 , Jing Lei 2 ’* and Yan Wen 3 

1 Graduate School of Education, Peking University, Beijing 100871, China; E-Mail: wgguo@pku.edu.cn 

2 

' Syracuse University, 336 Huntington Hall, Syracuse University, Syracuse, NY 13244, USA; 

E-Mail: ychenl29@syr.edu 

"5 

Education Institute of Beijing, 2 Wenxing St. Xicheng, Beijing 100120, China; 

E-Mail: wysmiler@126.com 

* Author to whom correspondence should be addressed; E-Mail: jlei@syr.edu; 

Tel.: 1-315-443-1362; Fax: 1-315-443-1812. 

Received: 13 December 2013; in revised form: 23 April 2014 / Accepted: 29 April 2014 / 

Published: 22 May 2014 


Abstract: With a large-scale online K-12 teacher professional development course as the 
research context, this study examined the effects of facilitating feedback on online learners’ 
cognitive engagement using quasi-experiment method. A total of 1,540 discussion messages 
from 110 learners (65 in the experimental group and 45 in the control group) were both 
quantitatively and qualitatively analyzed. Results revealed that facilitating feedback 
significantly impacted learners’ cognitive engagement: (1) the level of cognitive engagement 
presented in treatment group was significantly higher than that of control group; 

(2) cognitive engagement levels of original postings increased overtime in the treatment 
group, while decreased in the control group; (3) the difference in discussion quantities 
between two groups was not significant. Effective feedback strategies and the importance 
of facilitating feedback in creating quality online instruction were discussed. 
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1. Introduction 

With the fast advances in emerging web technologies, distance education has created greater 
opportunities for learners without constraints on geographic location, socio-economic condition, 
or time of their choosing [1,2], and is especially appreciated by adult learners who have fulltime work, 
family responsibilities, and multiple social obligations [3,4], According to Jonnasen, the cognitive tools 
embedded in the online environment provide learners with more experience of knowledge construction, 
self-directed inquiry, and collaborative learning [5]. This constructive enviromnent keeps alive the 
control-decentralized learning paradigm, in which learners have more opportunities to participate and 
contribute with their own voice [6,7]. In addition, many researchers advocate that distance education has 
great potential in reaching more students and reducing the cost ( i.e ., [8-10]). All these benefits bring 
great promise to distance education, and the demand for it will only continue to grow [11]. 

Although distance education has its strengths as mentioned above, it also faces some challenges due to 
physical separation [12]. This lack of physical presence often results in learners’ feeling of isolation, 
which is the primary reason for higher dropout rates in distance education [13]. In addition, 
increased chances of misunderstanding, difficulties in tracking learners’ learning, and more psychological 
distance between learners are some of the issues we encounter in distance education [14-17]. To address 
these issues, great attention has been paid to communication-orientated activities in online learning. 

Learners’ opportunity to interact with other participants is an essential component in 
instruction [5,18], and learning is more likely to occur when such interaction is enhanced [19]. 
In this regard, online discussion becomes an important component of the online learning 
experience [20-22]. Laurillard advocates this point and further suggests that, in distance education, 
the pedagogical benefits of technology rely primarily on how successfully it supports the dialogue 
between students, or between teacher and students [23]. When suitably organized, such dialogue could 
provide a communal space for learners to socially construct knowledge, in which they are encouraged 
to take responsibility for sustained and collective knowledge advancement [24]. Scandamalia and 
Bereiter indicate that knowledge building is more likely to occur in a community that is typified by 
discourse [25], This is because the dialogic mode provides more opportunities for learners to become 
co-creators of collective understanding and contributors of community knowledge base rather than passive 
infonnation receivers [26]. They make argument, diagnose misconception, negotiating perspectives, 
share personal experiences, challenge accepted beliefs, and demand reason for conclusions, and all of 
these are important cognitive presence in deep learning [27,28]. Meyer believes that the 
socio-cognitive dynamics of online discourse could establish a constructive, collaborative, intentional, 
and authentic learning experience for learners [29], in which not only could they gain a thorough 
understanding of the subject content, but also obtain critical thinking skills and inquiry capacities that 
are prerequisite for an independent thinker. 

However, the sole existence of discussion activities still cannot guarantee the effectiveness of 
distance instruction [28]. Researchers have identified some potential problems that might arise in 
online discussions. Meyer and Smith et al. found that learners are less motivated to participate in 
online discussions because written communication is more labor-intensive, and learners have to spend 
more time to read and respond to others’ postings [30,31]. Moreover, some studies reported low level 
of learning in online discussion [30,32,33]. Specifically, the discussions are merely of social nature 
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and stay at the superficial level. In some cases, the conversation is even not related to the 
pre-determined topic. [34,35], 

In situations like these, facilitating can play an important role in developing critical and in-depth 
discussions and improving online learning outcomes. According to Vandehaar, what matters in 
distance education is not the technology format, but the teaching and support from the instructor [36]. 
Feedback from the instructor/facilitator is one of the essential heuristics to promote learning [37]. 
Empirical evidence is shown by some research revealing that more effective learning occurs when 
learners are provided with instructors’ feedback [38-40]. Feedback could provide learners the outcome 
information regarding the correctness of their response [41], and additional information regarding the 
nature of outcomes and the qualities of their cognitive processing [39]. Fabro and Garrison reveal that 
instructor’s constructive critique into students’ thinking process significantly influences learners’ 
higher-order learning [42]. In an online course, the instructor’s presence by giving feedback has the 
potential to reduce the insecurity of lonely study [43], keep awareness of learning purpose [43], 
and improve self-confidence and self-esteem of students [45], 

To better understand the feedback pedagogy, feedback effects have been explored by scholars and 
adopted in various practices. For example, Nicol and Macfarlane-Dick proposed seven principles of 
good feedback practice for self-regulated learning [46]. Chai reported the feedback effects on 
knowledge acquisition in online learning [47]. Zumbach, Hillers, and Reimann examined the feedback 
mechanism in problem-based online learning [48], Butler and Winne studied the feedback in 
self-regulated learning from the theoretical view [39]. Effective feedback strategies in online instruction 
are identified including providing probing questions [45], clarifying the learning purpose/goal [40], 
encouraging positive motivational beliefs and self-esteem [46], provoking reflection [44,45], 
defining students’ roles/responsibilities, and encouraging students to explicitly confront others’ ideas [48]. 

A large body of research has explored the feedback effects on traditional classroom 
teaching ( i.e ., [39,49,50]). Some studies examined the feedback effects in online learning generally at 
test achievement level (i.e., [47,48,51]). The impact of feedback on students’ discussion activities, 
especially the specific intellectual aspects of discussion, in online learning environment has not been 
fully examined yet. Cognitive engagement is a concept proposed to describe such intellectual aspects 
in learning. In this study, we analyze the learner-generated data to examine the feedback effects on 
enhancing online learners’ cognitive engagement. 

Cognitive engagement speaks to “the extent to which learners are able to construct and confirm 
meaning through sustained reflection and discourse” in a learning community [20]. 
Early studies on online education mainly focused on how to create and maintain social interaction in 
online learning environment (i.e., [52,53]), gradually building a better understanding of social 
interaction in online learning [54]. Moreover, research found that social interaction by itself is not 
sufficient to reflect the quality and depth of learning [44]. Actively cognitive engagement that deepens 
the meaning of the educational experience is also very important [55,56]. Zhu points out that, 
even in social learning situation, the quality of learning still primarily depends on the levels of 
cognitive engagement of individual learners [57]. Thus, in this study, we focus our attention on the 
cognitive engagement of online learners, and use it to indicate the quality of online learning. 

In the present context, the facilitating feedback is the information learners receive from the 
instructor/facilitator about their discussion, with which learners can “confirm, add to, overwrite, tune, 
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or restructure” their ideas and understandings [39]. In this study, the online facilitators qualitatively 
grade the content of learners’ postings. Formative comments are given to maintain idea 
exchanges/improvement, keep conversations on the right track, promote deeper inquiry, and help 
learners to link practices to theoretical principles. By analyzing the data collected from online 
discussion forum, we intended to answer the question: What are the facilitating feedback effects on the 
level of cognitive engagement exhibited by learners in asynchronous online discussion? 

2. Experimental Section 

2.1. Research Setting 

This study was conducted in a National K-12 Schoolteacher Online Training Project in China [58], 
which was developed to teach K-12 teachers to integrate educational technologies into their classrooms. 
The online course in this study was run on Moodle [59], and was consisted of eight modules. Each module 
was open for a duration of three to five days. All learners were required to participate in three types of 
activities: reading and quiz, discussion, and hands-on practice [60]. 

The core of this course was the asynchronous online discussion activity. In each module, 
a discussion specification, including discussion topic, cases stories, orientation for preparation, 
and rules of participation, was provided to learners. With this specification, learners were guided to 
read the case stories and contribute their ideas and opinions afterwards. They were encouraged to 
develop their ideas based on the corresponding case stories and the module-learning theme. 
In addition, all learners were assigned to groups of 7-10 partners. Usually, learners with similar 
professional background were assigned to the same group to share teaching experience with each other. 
Learners were allowed to browse on all postings in the discussion forum, and required to at least respond 
to the postings in their own group. In order to create a learning community of being synchronous at 
model-level and asynchronous at posting-level [60], all learners were required to post at least one 
original posting, read other members’ postings and reply to at least two postings in each module. 

2.2. Participants 

A total of 110 distance learners of two classes in this online training program participated in this 
study. These distance learners were all K-12 in-service teachers in China who taught various subjects 
ranging from Math, Science, Chinese, History, Art, to Computer technology. The demographic data 
shows that all learners were adults, mostly over 30 years of age, and they all had a bachelor degree or 
above. In addition, all learners had basic computer skills, such as browsing online, 
uploading/downloading materials, and searching resources, yet none of them had any prior online 
training experience. Learners of the first class (n = 65) served as the experimental group, and learners of 
the second class (n = 45) constituted the control group. The first class received the training in May 2007, 
and the second class in January 2008. The training in both classes lasted for one month in total. 

2.3. Study Design 

A quasi-experiment design was used in this study. Both groups took the same training course, 
used the same learning materials, and participated in the same activities in eight modules. Particularly, 
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in the discussion activities, the two groups were provided with the same readings and the discussions 
were initiated by the same questions. The only difference between the experimental class and the 
control class was whether or not the facilitating feedback was available. 

For the experimental class, one online tutor was assigned to facilitate students’ online discussions 
by scoring and giving feedback to each posting. Such formative feedback followed these principles by 
encouraging learners to: (1) share their own teaching experience or use authentic information coming 
from their local schools or organizations; (2) revise and improve their arguments in the postings; 

(3) use appropriate tenninologies and concepts, and to relate their ideas with principles and theories; 

(4) reflect on their own learning; (5) have positive motivational beliefs and self-esteem. Within 24 h 
after the learner submitted his/her post, the tutor graded their postings using rubrics listed in Table 1. 
In this course, the tutor used the comments as the personal response to the learner’s discussions and 
contributions. At the same time, giving comments was the strategy to foster learners’ deeper thinking 
and reflection. In the comments, the tutor pointed out the strength and weakness of learners’ postings, 
offered guidance on discussion requirements and how to improve their current perfonnance, 
and employed probing strategies to move learners to higher-level cognitive engagement and to ensure 
continual cognitive development. The feedback was of formative nature. Learners were encouraged to 
revise their old postings and improve their discussion performance based on the comments given by 
the tutor. In order to motivate learners to re-approach discussion in a deeper way, the tutor would 
review and re-grade learners’ revised postings and gave bonus points to the new postings that 
demonstrated richer understanding and deeper thinking. The grade and the comments were provided in 
“score” and “teacher comment” boxes respectively as shown in Figure 1, and they were only open to 
the specific individual learner, and could not be viewed by others. 

In addition, during the online discussion activity, the tutor employed feedback strategies, 
such as praising, encouraging, and appropriate interruptions, to help learners actively engage in the 
conversations. Postings considered as excellent would be selected by the facilitator and recommended 
to all learners. 

The control class, by contrast, had no facilitator during the online learning process. After each 
module’s training, learners’ postings in the discussion forum were automatically graded by the 
learning manage system in tenns of the posting’s length and learners’ logging times. For example, 
two 300-word postings were graded the same score without examining the content. 


Table 1. Online discussion rubrics. 


Grade Level 


Excellent 
(35-40 points) 


Good 

(25-34 points) 

Acceptable 
(1-24 points) 
Incomplete 
(0 point)_ 


Rubric Explanation 

Express opinions in a clear and concise manner with strong connection with the given topics or readings 

Have made a logical argument with premises, reasons, and conclusion 

Analyze learning goals and learning activities with principles or theories 

Use personal teaching experience or external resources to support the important points 

Express opinions in a clear manner with being related with the given topics or readings 

Have made a logical argument with premises, reasons, and conclusion 

Analyze learning goals and learning activities with principles or theories 

Express opinions but without explanation 


Not his/her own opinion but plagiarizes others’ idea or work 
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Figure 1. Teacher score and comment interface for discussion postings. 


Comparison of Chinese and American Education: Concepts Bring Difference 
Published by *** Date: 

I am deeply touched by the two designs of history courses with distinct styles: 

Firstly, I think the two courses have the identical teaching goals. However, the history 
course of American is designed in a novel manner and closely related with students’ life. 
Focusing more on the cultivation of emotional engagement of students, the history courses 
design of China has clear three-dimensional teaching goals, and highlights cognitive 
development of students. 

Secondly, in my opinion, there is some difference between the two course in methods 
used to achieve the goals. The history course of America fully embodies the constructive 
learning theory and the humanism learning theory, while the history course of China is 
dominated by cognitivism learning theory. We should reflect on and learn something from 
such a difference. 

In Brief, in comparison of fundamental education of China and America, it is difficult to 
discriminate which is better. We hope all the school teaches could maintain our traditional 
advantages, borrow benefits from others, and conduct K-12 education down-to-earth. 

Score Com ments _ 

Which traditional advantages to maintain? How to borrow 
from other education culture? Please specify this point 
_ using your own teaching experience or stories. 


In order to explicitly demonstrate the facilitating condition in the treatment group, the comparison 
of discussion facilitating between two groups is illustrated in Table 2. 


Table 2. Comparison between treatment group and control group. 


Feedback Intervention 

in Two Groups 

Treatment Group 

Control Group 


Postings were graded by the tutor based on the 
content of the posting. 

Postings were graded automatically 

Grading criteria 

by the online learning system based 
on the length of posting. 

Comments 

Comments were given by the tutor within 24 h. 

The tutor respond by providing hints, probing 

No comments were given, no discourse 

questions or encouragement, reminding of 
overlooked steps, and directing learners’ attentions. 

development strategies are employed. 


Nature of feedback 

Formative. Within the three-day-length module, 
students were allowed/encouraged to 
revise/improve/reflect on their postings according 

to tutor’s comments. 

Summative. No suggestion was given 
to learners, and they do not have to 
improve or reflect on their postings. 


2.4. Measurement: Cognitive Engagement Level 

The Framework for Reflective Pedagogical Thinking developed by Sparks-Langer and Simmons 
was used as the coding scheme to assess learners’ cognitive engagement level. This framework was 
developed to assess schoolteachers’ ability to use concepts and principles to explain teaching activities 
and classroom events [59]. It has identified seven levels of thinking, based on Gagne’s hierarchy of 
thinking [61] and Van Manen’s idea of critical reflection [62,63]. 

As shown in Table 3, in this study, we focus on the five thinking levels used in this framework 
(the levels “No descriptive language” and “Explanation with consideration of ethical, moral, 
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and political issues” were not included). Rubrics were also developed to assess the cognitive levels of 
replying postings (see Table 4). Examples of each level were also provided in the two tables. 

A single posting was defined as the unit of analysis. We coded both the original and replying 
postings in the discussion forum, and identified the level of cognitive engagement of each one by using 
the indicators in coding schemas. 

Table 3. Cognitive coding schema for original posting. 


Cognitive Detailed description 
Level 

, Simple, layperson 

Level 1 


Examples 


Level 3 


I used to train teachers in our school, and I like using Powerpoint best. 


Level 2 

Events labeled with 
appropriate terms. 

We put too much emphasis on knowledge acquisition but ignored their critical 
thinking and their individual learning/thinking experience. 

Level 3 

Explanation with 
tradition or personal 
preference given as 
the rationale. 

We should not simply teach our teachers to use technology’, but rather we need to 
teach them how to integrate technology’ into their classrooms. Just like we 
always did in our own teaching, we want our students to not only acquire the 
knowledge, but also to apply the knowledge. 

Level 4 

Explanation with 
principle/theory given 
as the rationale. 

This is an effective way, because the performance assessment method is used. 

This method use “task" to motivate students, and measures what the students can 
do during the process. 

Level 5 

Explanation with 
principle/theory and 
consideration of 

context factors. 

Instructional media is rich in our life, but we do not have to be limited to using 
high-tech. All the tools, no matter multi-media or not, if appropriately used, 
could also improve our teaching. Recently when I taught the concepts of Angle 
and Edge in a 2nd grade math class in a rural area. I was told that no computers 
would be provided. Then I redesigned my instruction, and made an Angle model 
using stiffpaper and nails.... 

Table 4. Cognitive coding schema of replying postings. 

Cognitive 

Level 

Standard 

Examples 

Level 1 

Simple agreement or 
disagreement to the 
original postings. 

Strongly agree with you! 

Level 2 

Explanation with tradition, 
personal preference, or 
principle/theory given 
as the rationale. 

I agree with all you guys. New media is a “truck to deliver the cargo ”. 
However, we also need to have the ability’ to control the “truck”, 
and make this truck’s features fit into our teaching styles. Then we can 
have an effective instructional design. 


Explanation with tradition, 
personal preference, 
or principle/theory given 
as the rationale, 
and consideration of 
context factors. 


I agree with you. Technology’ based teaching is a good way to improve 
students ’ learning. But, we need to make sure that the technology’ is 
integrated appropriately. There was a lesson competition in my school, 
and we were required to use technology’ to teach. Some teachers just 
simply changed their materials to electronic version but did not pay 
attention to the pedagogical things, the results turned out to be not good. 
I know a very’ good philosophy teacher that he did not use any 
technology’. He just used role play games to teach students abstract 
philosophical concepts. His teaching was very’ successful. 
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2.5. Data Collection and Analysis 

There were eight modules in total in this online course, with the online discussion activity in each 
module. However, the discussions in module 1, 4, and 8 were not directly associated with the learning 
content, but were about social interaction ( i.e ., Icebreaking) and course evaluation. Thus, in the data 
collection, we excluded the postings in these three modules and finally collected 1,540 postings from 
the other five modules in the discussion forum. 

These threaded text-based postings were the main data source in this study. Content analysis 
method was applied to qualitatively analyze the content of postings aiming at providing a richer view 
on learners’ cognitive engagement and the progression of the discussion. The researchers carefully 
studied all the transcripts, and used the coding schemas in Tables 3 and 4 to identify learners’ 
cognitive engagement and grouped them into different levels. To increase the inter-rater reliability, 
negotiated coding approach was employed. The researchers coded the discussion transcripts, and later 
actively discussed and revised until all researchers arrived at a final agreement [64]. 

3. Results and Discussion 

Analyses of discussion transcripts of 110 learners reveal that: (1) no significant difference in the 
quantity of discussion postings between two groups were found; (2) in each module the cognitive 
engagement levels presented in original and replying postings were much higher in experimental group; 
(3) cognitive engagement levels of original postings was progressively increased overtime in the 
experimental group, while it gradually decreased in the control group. 

3.1. Comparison of the Quantity of Postings 

As shown in Table 5, the experimental group posted slightly more messages in the online discussion. 
On average, each learner in the experimental group posted 1.06 original messages per week, while the 
control group posted 1.02; each learner in experimental group posted 1.77 replying messages, 
while the control group posted 1.71. 

Although the data showed that the experimental group slightly outperfonned control group in the 
quantity of discussion, the difference was not significant. Two reasons might account for this finding: 
(1) the discussion rule of “at least one original posting and two replies” ensured the control group 
learners’ participation; (2) mandatory grouping and students being told to discuss only within their 
own group limited their interactions with peers in other groups. 

In each module, learners were required to contribute at least one original posting and reply to at 
least two people. However, we noticed that the actual numbers of original postings by each student per 
module in control group and experimental group were 1.02 and 1.05, respectively, and the replying 
ones were 1.76 and 1.77. This indicated that, on average, learners in this course were much more 
rubric-directed than self-directed. It seemed that they posted messages just to meet the minimum 
requirement in terms of the posting quantity. Once they reached the minimum requirement, the inquiry 
process ceased. To help learners recognize their ideas in discussion could be continually improved is 
important for driving them to a deeper level of inquiry [24]. Further study is very much needed to 
understand how to motivate learners to make more extensive use of online discussion without being 
limited to the minimum requirements. 
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Table 5. Distance learners’ participation in online discussions. 


Subjects 

Total Number 

Total Number 

Total Number of 

Total Number of 

Average Number of 

of Learners 

of Postings 

Original Postings 

Replying Postings 

Postings per Learner 

Control group 

65 

904 

333 

571 

13.9 

Experimental group 

45 

636 

238 

398 

14.1 


3.2. Comparison of the Level of Cognitive Engagement 

The cognitive engagement in original postings. As Figure 2 illustrated, the starting points of two 
groups were very close to each other, which indicated that these two groups demonstrated almost 
equivalent cognitive level in online discussions before the facilitating feedback intervention. 
However, after facilitating feedback was provided to the experimental group, the difference between 
two groups grew as time went by. Moreover, the cognitive level of online discussions in the 
experimental group rose overtime, while the control group showed a trend of gradual decline. 


Figure 2. The cognitive level curves of original postings over time. 



In addition, we noticed that, the experimental group’s cognitive level of discussions in every 
module was higher than that of the control group. On average, the mean cognitive level of 
experimental group was 3.84 and that of control group was 2.85. An independent sample T test 
showed that the difference of cognitive level of discussion between two groups was significant 
(t= 12.16,/? <0.001). 

The results indicated that, with the facilitating feedback for their online discussions, the cognitive 
engagement level in the experimental group was significantly, increasingly, and stably higher than that 
of the control group, which had no facilitating feedback. In other words, facilitating feedback had 
evidently improved the cognitive engagement of learners in their online discussions. Four factors 
might be responsible for this: (1) With the constant feedback and guidance from the facilitators, 
the learners in the experimental group could in a clear way develop their skills on connecting practice 
with theories, reflecting on cases or teaching events, and justifying their arguments. This was also 
shown in the raising trend of their cognitive engagement level; (2) Learners in the experimental group 
were given the opportunities to revise and refine their ideas/postings, which stimulates or maintains 
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learners’ continually cognitive engagement and development. (3) Facilitator’s presence could serve as 
an external stimulus to generate more monitor pressure on learners, and they were more motivated and 
might perform better in the discussion; (4) The interactions between the facilitator and learners could 
reduce learners’ feeling of loneliness and a sense of isolation, and a sense of community or belonging 
could make them contribute more to the online discussions. While in the control group, the feeling of 
freshness for online discussions might easily wane over time. Without the guidance and 
encouragement from facilitator, it could become harder for them to maintain their enthusiasm and to 
master the effective strategies for reciprocal communication and critical thinking in the online 
discussions. In addition, the lack of substantive feedback reduced the opportunities and motivations to 
rethink and further revise their ideas. 

The cognitive engagement in replying postings. As shown in Figure 3, the cognitive engagement 
level of replying in the experimental group was consistently higher than that in the control group. 
Generally, the cognitive levels of most replying messages in the experimental group were above 2, 
while those in the control group were below 1.5. This indicated that when responding to others’ 
postings, learners in the experimental group presented not only their statement, but also included 
detailed evidences to support their argument. However, most learners in the control group simply 
voiced their agreement or disagreement without any further analysis. 

Figure 3. The cognitive level curves of replying. 



The discrepancy between two replying curves might be caused by two factors: the cognitive level of 
the original postings and the intervention of facilitating feedback. The learners who showed deeper 
thinking in the original postings were more likely to demonstrate higher level of cognitive engagement 
in their later responses to their peers’ comments. In additino, the ideas and opinions in high quality 
original messages were also more likely to inspire in-depth thinking and further discussion. 
Again, in the experimental group, the facilitator constantly reminded learners to provide evidence 
(cases, their own experience, etc.) to support their argument, and guided learners to enrich their ideas and 
l ink theories to their own experience. Such facilitating feedback helped them to reach level two or three. 

It is interesting to find learners in the two groups posted almost the same number of messages but 
demonstrated significantly different cognitive engagement levels. The number of messages, 
in some extent, demonstrates the degree/frequency of social interaction among learners. In other words, 
learners’ cognitive engagement performance was not necessarily related with the number of interactions. 
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This result is consistent with the findings of Kanuka and Anderson’s study [65]. They coded the 
students’ online discussion transcripts, and found that considerable interactions occurred but a low 
level of cognitive engagement was detected. Garrison et al. pointed out that, in online discussion, 
quantity of interaction is not enough, and social presence by itself is not sufficient to reflect the quality 
and depth of learning [44]. Thus, our data offers a clue for assessing the quality of online discussion, 
and also shows that the posting quantity hardly reflects the quality of discussions. To pursue a 
meaningful online discussion directed to cognitive outcomes, qualitative dimension characterized by 
learners’ cognitive engagement should be stressed more than quantitative measures. This point is 
consistent with the conclusions of several previous studies [44,65-67]. 

In addition, we also noticed that very few original/replying postings, in both group, had reached the 
top level of cognitive engagement in the online discussion. Could online discussion be an appropriate 
activity to create higher-order cognitive engagement? Many empirical distance education studies have 
investigated this issue, and higher-order cognitive engagement, for example, idea integration, 
testing, and application, are reported in computer-mediated communication than face-to-face 
conversation (i.e., [27,68]). The precise and permanent nature of written communication made it a 
different learning experience for online learners. The ideas contributed by learners are all recorded in 
written form, which enable learners to reflect on their previous understanding. In addition, 
when expressing using deliberate and explicit written words, learners are encouraged to carefully 
organize and justify their argument [69]. In this study, the reasons for the absence of top level 
cognitive engagement might be associated with the learning culture of “knowledge transmission 
lecture” in China. With many years of learning in such paradigm, K-12 teachers got more used to be 
passive information receivers rather than active contributors. When this online teachers training 
program was launched in 2007, many online leamers-Chinese K-12 teachers-didn’t know how to 
“speak” in online discussion, they just posted paragraphs of learning theories on the discussion boards, 
but seldom shared their own teaching stories and problems, or expressed their opinions in a logical 
and critical way. 

4. Conclusions 

This study has investigated the effects of facilitating feedback on discussions within an online program 
of K-12 in-service teacher training in China. By quantitatively and qualitatively analyzing the content of 
1,540 postings of two classes, we have investigated the effects of facilitating feedback on learners’ 
discussion perfonnance in terms of cognitive engagement. The findings emerged from this study include: 

(1) no significant difference of the quantity of postings was found between control and experimental group; 

(2) facilitating feedback positively influenced the level of cognitive engagement presented in the discussion 
within each module; (3) with facilitating feedback, cognitive engagement levels of original postings 
increased overtime; while without it, the cognitive engagement levels decreased. 

Based on the findings mentioned above, we drew the conclusion that such facilitating feedback has 
significant effects in enhancing learners’ cognitive engagement in online discussions. This kind of 
feedback strategies include: giving encouraging infonnation, asking thought-provoking questions to 
nurture in-depth and reflective discussion, and appropriately interrupting their dialog to keep discussion 
on track. Our results reinforced the conclusion of previous studies in that the quantity of social 
interactivity ( i.e ., postings) does not reflect the depth of online discourse (i.e., cognitive engagement). 
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Unfortunately more detailed demographic information about the participants was not available, 
for example, the years of their teaching experience, the ratio of male and female, and specific age 
distribution. Thus, we had no ways to depict a richer picture of our participants. Knowing about the 
background information is helpful in thoroughly understanding how the formative comments 
facilitating works in the specific context of this study. 

Future research investigating effective facilitating feedback strategies to promote critical online 
discussion, feedback’s role in changing online learning process, timing of such facilitating feedback, 
the feedback mechanisms in online discussion, and how feedback effects interact with learners’ 
characteristics (e.g., motivation, knowledge base) would be promising directions. 
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